Task-aware Scheduling Algorithms

A common pattern in the architectures of modern interactive web-services is that of large request fan-outs, where even a single end-user request (task) arriving at an application server triggers tens to thousands of data accesses (sub-tasks) to different stateful backend servers. The overall response time of each task is bottlenecked by the completion time of the slowest sub-task, making such workloads highly sensitive to the tail of latency distribution of the backend tier. To address such challenges, we present BetteR Batch (BRB), a system that carefully schedules requests in a decentralized and task-aware manner. In doing so, we improve performance predictability of data stores in the presence of large request fan-outs. We show via simulations that our proposed design improves latencies over the state-of-the-art by a factor of 2.